2 research outputs found
An Effective Meaningful Way to Evaluate Survival Models
One straightforward metric to evaluate a survival prediction model is based
on the Mean Absolute Error (MAE) -- the average of the absolute difference
between the time predicted by the model and the true event time, over all
subjects. Unfortunately, this is challenging because, in practice, the test set
includes (right) censored individuals, meaning we do not know when a censored
individual actually experienced the event. In this paper, we explore various
metrics to estimate MAE for survival datasets that include (many) censored
individuals. Moreover, we introduce a novel and effective approach for
generating realistic semi-synthetic survival datasets to facilitate the
evaluation of metrics. Our findings, based on the analysis of the
semi-synthetic datasets, reveal that our proposed metric (MAE using
pseudo-observations) is able to rank models accurately based on their
performance, and often closely matches the true MAE -- in particular, is better
than several alternative methods.Comment: Accepted to ICML 202
The ACROBAT 2022 Challenge: Automatic Registration Of Breast Cancer Tissue
The alignment of tissue between histopathological whole-slide-images (WSI) is
crucial for research and clinical applications. Advances in computing, deep
learning, and availability of large WSI datasets have revolutionised WSI
analysis. Therefore, the current state-of-the-art in WSI registration is
unclear. To address this, we conducted the ACROBAT challenge, based on the
largest WSI registration dataset to date, including 4,212 WSIs from 1,152
breast cancer patients. The challenge objective was to align WSIs of tissue
that was stained with routine diagnostic immunohistochemistry to its
H&E-stained counterpart. We compare the performance of eight WSI registration
algorithms, including an investigation of the impact of different WSI
properties and clinical covariates. We find that conceptually distinct WSI
registration methods can lead to highly accurate registration performances and
identify covariates that impact performances across methods. These results
establish the current state-of-the-art in WSI registration and guide
researchers in selecting and developing methods